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Executive Summary 

NASA prefers to land the space shuttle at Kennedy Space Center (KSC). When weather conditions violate 
Flight Rules at KSC, NASA will usually divert the shuttle landing to Edwards Air Force Base (EAFB) in Southern 
California. When this happens, forecasting surface winds at EAFB is a challenge for the Spaceflight Meteorology 
Group (SMG) forecasters due to the complex terrain that surrounds EAFB. One particular phenomenon identified by 
SMG that makes it difficult to forecast the EAFB surface winds is called ‘‘wind cycling”. This occurs when wind 
speeds and directions oscillate among towers near the EAFB runway leading to a challenging deorbit burn forecast 
for shuttle landings. Accurately forecasting the winds with at least 2 hours lead time along the runway is crucial in 
supporting a landing decision. The large-scale numerical weather prediction models cannot properly resolve the 
wind field due to their coarse horizontal resolutions, so a properly tuned high-resolution mesoscale model is needed. 
The Weather Research and Forecasting (WRF) model meets this requirement. 

The Applied Meteorology Unit (AMU) was tasked to determine the skill of different WRF model 
configurations in forecasting wind cycling cases at EAFB. The AMU compared model output of wind speed and 
direction from two WRF model dynamical cores, two “hot-start” initialization options of the WRF model and three 
physics options. The various combinations of these options resulted in a total of 42 model runs for seven wind 
cycling case days and 14 model runs for two null case days. The initialization options are three-dimensional weather 
{ Dialys is systems that integrate mu lt ipl e meteor ologica l data sources into one analysis over the user's domai n of 
interest. These analysis systems allow mesoscale models to benefit from the addition of high-resolution data sources 
in their initial conditions. 

The goal of this work was to assess the different model options and to determine which configuration best 
predicted surface wind speed and direction at EAFB. To do so, the AMU compared the WRF model performance 
using both hot start initializations with both dynamical cores and compared model performance while varying the 
physics options. The model output was compared to the wind tower data from the seven wind-cycling and two null 
case days. 

Based on both subjective and objective analyses comparing the model forecast data to observations, the AMU 
found the Advanced Research WRF core and Mellor-Yamada-Janjic planetary boundary layer scheme performed 
better than the other model configurations. The model did not produce a better wind forecast on null days vs. wind 
cycling days. However, the model was able to differentiate between wind cycling days and null cases which would 
provide added value to the SMG forecaster’s shuttle deorbit bum landing forecast. 
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1. Introduction 

Occasionally, the space shuttle must land at Edwards Air Force Base (EAFB) in Southern California when 
weather conditions at Kennedy Space Center (KSC) violate Flight Rules (FR). However, the complex terrain in and 
around EAFB makes forecasting surface winds a challenge for the Spaceflight Meteorology Group (SMG). In 
particular, “wind cycling” cases, in which the wind speeds and directions oscillate among towers near the EAFB 
runway, present a challenging forecast problem for shuttle landings. An accurate depiction of the winds along the 
runway is crucial in making the landing decision. Global and national scale numerical weather prediction models 
cannot properly resolve the wind field due to their coarse horizontal resolutions, so a properly tuned high-resolution 
mesoscale model is needed. The Weather Research and Forecasting (WRF) model meets this requirement. 

The goal of this work was to assess different WRF model options and to determine how well the model could 
predict surface wind speed and direction at EAFB and if one model configuration performed better than others. The 
data presented in this report shows many interesting phenomena that were not investigated in the interest of focusing 
on the SMG goal of model capability and time constraints to complete the work. Given the time constraints and the 
observation that both wind speed and direction indicated cycling during an event. The Applied Meteorology Unit 
(AMU) chose to focus on assessing model performance forecasting wind speed. 

The AMU investigated the WRF model’s two dynamical cores: 1) the Advanced Research WRF (ARW) and 2) 
the Mon-hydrostatic Mesoscale ModeT(NMM), There-are also two options for-atihoUstart” ini tialization of the WRF -~ 
model: I) the Local Analysis and Prediction System (LAPS; McGinley 1995) and 2) the Advanced Regional 
Prediction System (ARPS) Data Analysis System (ADAS; Brewster 1996). Both LAPS and ADAS are three- 
dimensional weather analysis systems that integrate multiple meteorological data sources into one analysis over the 
user’s domain of interest. These analysis systems allow mesoscale models to benefit from the addition of high- 
resolution data sources in their initial conditions. In addition to model core and initialization options, there are many 
different model parameterization options within each core. Having a series of initialization options and WRF cores, 
as well as many options within each core, provides SMG with considerable flexibility as well as decision-making 
challenges. The goal of this study is to assess the different model configurations and to determine which 
configuration will best predict surface wind speed and direction at EAFB. To accomplish this, the AMU compared 
the WRF model performance using ADAS versus LAPS for the ARW and NMM cores and compared model 
performance while varying the physics options. 


1.1 Background Information 


The location of EAFB (Figure 1) is on the western edge of the Mojave Desert. It is surrounded by mountains on 
three sides: the San Bernardino Mountains to the southwest, the San Gabriel Mountains to the south, and the 
Tehachapi Mountains and southern Sierra Nevada to the west and northwest. The base is located next to Rogers Dry 
Lake whose hard surface acts as an extension to the EAFB runways. There are twelve wind towers located in and 
around the EAFB runways. Figure 2 shows the locations of the towers along the EAFB runway complex. 



Figure 1. Location of EAFB in relation to surrounding mountainous 
terrain. Background image from MSN Live Search Maps. 
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Figure 3. Visible satellite image showing several mountain wave clouds on 30 January 2008 
near EAFB. The yellow circle indicates the location of EAFB. 


The complex terrain located in and around EAFB creates the erratic behavior of the wind speed and direction 
seen on the EAFB wind towers. Wind cycling events occur when there is an oscillation in wind direction and/or 
wind speed among the wind towers in the network near the EAFB runway complex. During these cycling events, the 
wind speed and direction reported from the towers near the concrete runway (Towers 44, 220, 224) are noticeably 
different than that reported from towers near the lakebed runway (Towers 154, 230, 234). These events usually last 
from 90 minutes up to 4 hours or longer and most often occur when the prevailing wind is from the northwest or 
west-northwest. 


The wind cycling events at EAFB are thought to be caused by mountain waves created as winds from the 
northwest flow over the Tehachapi Mountains. Figure 3 shows several mountain wave clouds that developed near 
EAFB on 30 January 2008. On this day, the wind along the concrete runway fluctuated from westerly and relatively 
lighter to northwesterly and stronger. The winds along the lakebed were northwesterly and strong during the wind 
cycling event. 


Figure 2. Wind tower locations on EAFB. The towers along the concrete and lakebed 
runways are indicated by arrows. Background image from MSN Live Search Maps. 
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1.2 Report Format and Outline 

This report presents the findings from a one-year study of model sensitivities for predicting wind cycling events 
at EAFB. This analysis examined different model cores, initializations, and physics options to determine the impact 
on model skill. Two local data analysis systems that ingested satellite data, radar data, and surface observations 
across southern California were used to initialize a mesoscale model. Model skill was assessed using both a 
subjective and objective verification that compared forecast to observed wind speed and direction from the local 
EAFB wind towers. Section 2 describes the data and methodology including an example of a wind cycling case at 
EAFB, the different model cores, model initialization options, and the data ingested. The results of the subjective 
and objective analyses are presented in Sections 3 and Section 4, respectively. Section 5 summarizes the report and 
provides recommendations on the best mesoscale model configuration for operational use based on this study. 
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2. Data and Methodology 

The important aspects of this study were the choice of wind cycling case days, the model configuration, and the 
data used to initialize the models. The candidate wind cycling days chosen occurred between December 2006 and 
July 2008. Comparisons were made between the ARW and NMM core initialized with ADAS and LAPS, as well as 
with different physics parameterizations. Both subjective and objective analyses were used to verify the forecasts. 

2.1 Wind Cycling Candidate Days 

The period of record for choosing wind cycling candidate days was December 2006 through July 2008. SMG 
provided the AMU with the dates of seven wind cycling cases. Six of the seven were strong wind cycling cases and 
one, the 30/31 July 2008 case, was marginal in that the fluctuations in wind speed and direction were smaller than 
for all other wind cycling cases. Two null, or non-wind cycling, days were also randomly chosen from the available 
data. Table 1 lists the candidate days, the start of the wind cycling event, and the end of the event. 


Table 1 . List of all wind cycling candidate days, null cases, start of the wind 
cycling event, and the end of the event. 

Candidate Day 

Start of Event 

End of Event 

22 Dec 2006 

0900 UTC 

1800 UTC 

30 Jan 2008 

1030 UTC 

1330 UTC 

14 Feb 2008 

0300 UTC 

0700 UTC 

5 Mar 2008 

0000 UTC 

1000 UTC 

4 Jun 2008 

1100 UTC 

2000 UTC 

7 Jun 2008 

0200 UTC 

1300 UTC 

30/3 1 Jul 2008 

1 900 UTC 

0100 UTC (31 Jul) 

9 Jun 2007 

NULL CASE 

1 Nov 2007 

NULL CASE 


Figure 4 shows a time series of wind speed and direction at Towers 44 and 234 from a wind cycling event that 
occurred on 14 February 2008. Inspection of Figure 4 reveals a wave-like behavior in the time series between 0300 
and 0700 UTC (denoted by the blue vertical lines in Figure 4) for wind speed and direction at Tower 44 (top), which 
is located near the concrete runway. During this time there was an oscillation between southwesterly and 
northwesterly wind, as well as a 5 to 15 kt change in wind speed, that occurred approximately every half hour. The 
wind at Tower 234, near the lakebed runway, did not exhibit this wave-like behavior. 
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Figure 4. Wind direction (degrees) and wind speed (kts) for Tower 44 (top) and Tower 234 (bottom) 
on 14 February 2008 from 0000 to 12000 UTC. The green line is the wind direction, the red line is the 
peak wind and the blue line denotes the steady-state wind. Wind cycling occurs in the time period 
between the vertical dashed blue lines. 


2.2 Model Core and Initialization Options 

For this study, the AMU employed both the ARW and NMM cores of the WRF model. The ARW core was 
developed primarily at the National Center for Atmospheric Research (NCAR) and is a fully compressible, non- 
hydrostatic mesoscale model with a hydrostatic option. It consists of a mass-based hydrostatic pressure terrain 
following coordinate, Arakawa C-grid staggering for the horizontal grid, time-split integration using a third order 
Runge-Kutta scheme with a small step for acoustic and gravity wave modes, and up to sixth order advection options 
in the horizontal and vertical (Skamarock et al. 2005). There are also full physics options for microphysics, 
planetary boundary layer, cumulus parameterization, radiation, and land surface schemes (Skamarock et al. 2005). 
The NMM core is also a fully compressible, non-hydrostatic mesoscale model with a hydrostatic option (Janjic et al. 
2001, Janjic 2003a, b). It consists of a hybrid sigma-pressure, terrain following vertical coordinate, Arakawa E-grid, 
a forward-backward time integration scheme, a second order advection option in the horizontal and vertical, and 
conservation of energy and enstrophy (Janjic 1984). Most physics packages available to the ARW have not been 
tested with the NMM and therefore, the physics options for the NMM are more limited than for the ARW. 

A “hot-start" initialization of the WRF model can be made using LAPS or ADAS. LAPS is a data assimilation 
tool that uses numerous meteorological observations, such as satellite data, radar data, and surface observations, to 
generate a three-dimensional representation of the atmospheric forcing fields, such as wind speed and direction, 
surface temperature and pressure, relative humidity, precipitation and cloud cover (McGinley et al. 1991; Albers 
1995; Albers et al. 1996; Birkenheuer 1999; McGinley 1995). It includes a wind analysis and a three-dimensional 
cloud analysis, which are needed for the LAPS hot-start initialization. The LAPS cloud analysis is designed to create 
consistency with all data and the typical meteorology of clouds by combining data from infrared and visible satellite 
data, three-dimensional LAPS radar reflectivity derived from the full-volume radar data, and the LAPS three- 
dimensional temperatures (Albers et al. 1996). Fields derived from the cloud analysis include cloud liquid water, 
cloud type, cloud droplet size, and icing severity (Albers et al. 1996). However, in the latest versions of LAPS, the 
option to initialize precipitation was turned off. Mr. Chris Anderson of the National Oceanic and Atmospheric 
Administration Earth System Research Laboratory Global Systems Division (GSD) indicated that at a 1-km grid 
spacing, initializing WRF with hydrometeors could create a large outflow that would degrade the forecasts. 
Consequently, GSD zeroed out the precipitation particles in the initialization, but left the cloud particles. Fie 
recommended keeping the precipitation initialization turned off if it was not an essential component of the research. 
Therefore, the AMU decided not initialize to precipitation since this study was not focused on convective 
forecasting. 

The ADAS system, developed by the University of Oklahoma, has two main components. The first is a Bratseth 
objective analysis scheme that evaluates pressure, wind, potential temperature, and specific humidity. The second 
component is a three-dimensional cloud analysis scheme that is used for the hot-start initialization (Zhang et al. 
1998). The ADAS cloud analysis is based on the LAPS cloud analysis with some modifications (Case et al. 2002). It 
uses surface observations of cloud cover and height, satellite data, and radar data to determine the cloud cover, cloud 
liquid and ice water, cloud type, rain/snow/hail mixing ratios, icing severity, in-cloud vertical velocity, cloud base 
and top, and cloud ceiling (Case et al. 2002; Zhang et al. 1998; Brewster 2002). 

2.3 Data Ingest 

Data ingested by the model through either the LAPS or ADAS analysis packages included Level II Weather 
Surveillance Radar- 1988 Doppler (WSR-88D) data from the Las Vegas, NV (KESX), Yuma, AZ (KYUX), Santa 
Ana Mountains, CA (KSOX), San Joaquin Valley, CA (KHNX), San Diego, CA (KNKX), and Los Angeles, CA 
(KVTX) radars (Figure 5), Geostationary Operational Environmental Satellites (GOES) visible and infrared satellite 
imagery, Meteorological Assimilation Data Ingest System (MAD1S; http://madis.noaa.gov/) data, and local EAFB 
wind tower data. The Level II WSR-88D data contained full volume scans of reflectivity at a resolution of 1° by 1 
km, radial velocity at 1° by 0.25 km, and spectrum width data at a 1° by 0.25 km (Fulton et al. 1998). These data 
were available every 4 to 6 minutes. The GOES- 12 visible imagery was available at a 1 km horizontal resolution 
every 15 minutes, and the infrared imagery was available at a 4 km horizontal resolution also every 15 minutes. 
Both visible and infrared imagery provided brightness temperatures to the analysis packages. Each initialization 
analysis used a different background model to create the first-guess background fields. The choice of background 
data for both LAPS and ADAS was based on the highest resolution model data currently supported in the latest 
versions of both analysis packages. The latest version of LAPS allows the use of the Rapid Update Cycle (RUC) 20- 
km model as background data, while ADAS supports the North American Model (NAM) 12-km data. 
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Figure 5. Locations of WSR-88D sites (yellow) used in creating the high-resolution LAPS and ADAS analyses. 
The black circle shows the location of EAFB. 


A suite of scripts written by the AMU formed the core for initializing the WRF model with ADAS output. The 
analysis package ingested satellite data in Man computer Interactive Data Access System (McIDAS) Area format 1 5 
minutes prior to model initialization time; raw, full volume radar data within 10 minutes of the model initialization 
time; and surface data from the hour of and hour prior to model initialization. The largest task in configuring LAPS 
was working with the ingest code. The ingest code can only be used with raw data that have the same configuration 
and format as GSD’s raw data, which is the Network Common Data Form (NetCDF). Therefore, all data files were 
converted to NetCDF format to be used within LAPS. Software to convert the WSR-88D data to NetCDF format 
was obtained from GSD. Scripts were written to convert raw wind tower data into a format ingestible by both LAPS 
and ADAS. Converting satellite data in McIDAS Area format to NetCDF required several steps. The data were first 
ported to the local Meteorological Interactive Data Display System (MIDDS) system where they were remapped to 
the Lambert Conformal projection using the IMGREMAP command. Next, the remapped data were run through a 
program that converted them to NetCDF format. This program is called AreaToNetCDF and is available from the 
Space Science and Engineering Center (SSEC) at the University of Wisconsin. All of the reformatted data files were 
then ingested into LAPS to create an initialization field for the model. 

2.4 Model Configuration 

Each model simulation was run at a 1-km horizontal grid spacing over EAFB and adjacent mountainous areas 
with 51 irregularly spaced, vertical sigma levels. Each run was integrated 12 hours and began one to three hours 
before the onset of the observed wind cycling event. Table 2 lists the WRF model run times for each of the 
candidate days. Several factors were considered in determining the grid spacing of the WRF runs: the average 
distance between the wind towers, the customer requirements, and the computational requirements to run a high- 
resolution model. Grasso (2000) notes that the theoretical minimum resolvable wavelength by a model is twice the 
horizontal grid spacing, but Walters (2000) suggests that as many as 10 grid points may be required for a more 
reasonable representation of the true solution. The AMU determined that a 2-km or less horizontal grid spacing 
would be sufficient to resolve the wind cycling events given the average distance of 8 km between the wind towers 
on the runway (44, 220, and 224) and the lakebed (154, 230, and 234) and taking into account the spacing of all the 
wind towers. However, customer requirements indicated that a 1.3-km or less grid spacing was desired. To 
determine the highest resolution that could feasibly be run in real-time, three different runs using 1.33, 1, and 0.66- 
km grid spacing were made to determine the optimal resolution for the WRF forecasts. The I -km run slightly 
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outperformed the 1.3-km run with respect to wind speed and direction forecasts. The model was then run with a 
0.66-km grid spacing, but it took more than twice as long to run as compared to the 1-km WRF forecast. Since this 
is not practical for real-time operations, the 0.66-km WRF run was not considered and the 1-km grid spacing was 
used. 


Table 2. List of WRF model run times for all wind cycling and null cases. 

Candidate Day 

Model Start Time 

Model End Time 

22 Dec 2006 

0600 UTC 

1 800 UTC 

30 Jan 2008 

0900 UTC 

2100 UTC 

14 Feb 2008 

0000 UTC 

1200 UTC 

4/5 Mar 2008 

2100 UTC (4 Mar) 

0900 UTC (5 Mar) 

4 Jun 2008 

0900 UTC 

2100 UTC 

7 Jun 2008 

0000 UTC 

1200 UTC 

30/3 1 Jul 2008 

1 800 UTC (30 Jul) 

0600 UTC (31 Jul) 

9 Jun 2007 

0600 UTC 

1800 UTC 

... 1 Nov 2007 ... 

0900, LUC- 

2100 UTC 


Boundary conditions for the WRF runs were obtained from the NAM model with a horizontal grid spacing of 
12-km. The NAM model domain is on a Lambert conformal projection with a horizontal domain size of 614 by 239 
grid points and 42 vertical levels. It produces an 84-hr forecast every six hours, or four times per day. This model 
was chosen for boundary conditions as it has the best resolution of all available datasets. 

Each candidate day consisted of six runs using different model configurations, for a total of 42 runs for the wind 
cycling case days and 14 for the null cases. The different configurations included: 

• LAPS-ARW with the Yonsei University planetary boundary layer (PBL) scheme and NCAR/Penn State 
Mesoscale Model Version 5 (MM5) similarity surface layer scheme (LAPSARWYonsei), 

• LAPS-ARW with the Mellor-Yamada-Janjic PBL scheme and ETA similarity surface layer scheme 
(LAPSARW^MYJ), 

• LAPS-NMM with the National Center for Environmental Prediction (NCEP) Global Forecast Systems 
(GFS) PBL scheme and NCEP GFS surface layer scheme (LAPSNMM_GFS), 

• LAPS-NMM with the Mellor-Yamada-Janjic PBL scheme and ETA similarity surface layer scheme 
(LAPSNMMJV1YJ), 

• ADAS-ARW with the Yonsei University PBL scheme and MM5 similarity surface layer scheme 
(ADASARWJYonsei), and 

• ADAS-ARW with the Mellor-Yamada-Janjic PBL scheme and ETA similarity surface layer scheme 
(ADASARWJVIYJ). 

All other physics parameters were the same for each ARW model run. The NMM runs did not use the same physics 
as the ARW runs since the ultimate goal of the task was to see if wind cycling could be simulated. Instead, the best 
and most widely tested choice of physics found in the literature for the NMM core were used. Table 3 lists the 
physics options used in both the ARW and NMM runs that were held constant. 


Table 3. List of the physics o 

otions used for each model run for both the ARW and NMM cores. 


ARW 

NMM 

Microphysical scheme 

Linet al. (1983) 

Ferrier (1994) 

Cumulus Parameterization 

None 

None 

Land surface option 

Noah Land Surface Model (Chen 
and Dudhia 2001) 

Noah Land Surface Model (Chen and 
Dudhia 2001) 

Shortwave radiation scheme 

Goddard (Chou and Suarez 1 994) 

GFDL (Lacis and Hansen 1 974) 

Longwave radiation scheme 

RRTM (Mlaweretal. 1997) 

GFDL (Fels and Schwarzkopf 1975; 
Schwarzkopf and Fels 1985, 1991) 
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3. Subjective Analysis 

The AMU completed a subjective analysis of the WRF forecasts for all candidate days. The goals of the 
subjective analysis were to determine 

• If the model was able to predict the timing and/or magnitude of the wind cycling events at the concrete 
runway towers by comparing the observed wind speed to the forecast wind speed and 

• If the model could provide the forecasters with an indication of whether or not a wind cycling event was 
likely to occur by assessing the model forecasts on wind cycling days and null case days. 


3.1 Model Performance at the Concrete Runway Towers 

Using the Grid Analysis and Display System software, the AMU extracted the mode! forecast winds for every 
model run at each grid point nearest to each of the three concrete runway towers (44, 224 and 220 as shown in 
Figure 2) for comparison. The three tower locations and nearest corresponding grid point used in the comparison are 
shown in Figure 6. The model grid was identical for the MYJ and Yonsei PBL schemes and is shown in Figure 6a, 
while the model grid was slightly different for the GFS PBL scheme and is shown in Figure 6b. 



Figure 6. Maps of EAFB showing the locations of the three concrete runway towers (red dots) and the 
model grid points (black dots) for the MYJ and Yonsei PBL schemes (a) and GFS PBL scheme (b). The 
grid point closest to each corresponding tower used in the analysis is shown within each green ellipse. 

The AMU plotted graphs of the observed steady-state wind speed at all three towers and overlaid the forecast 
steady-state wind speed from each of the model configurations for 42 wind cycling and 14 null case model runs. 
There are three graphs per tower per day that display the output from the different model configurations. These 
graphs included the following configurations: 

• LAPS ARW 

o Yonsei PBL scheme and MM5 similarity surface layer scheme 
o MYJ PBL scheme and Eta similarity surface layer scheme 

• LAPS NMM 

o GFS PBL scheme and GFS similarity surface layer scheme 
o MYJ PBL scheme and Eta similarity surface layer scheme 

• ADAS ARW 

o Yonsei PBL scheme and MM5 similarity surface layer scheme 
o MYJ PBL scheme and Eta similarity surface layer scheme 


An example of a wind cycling event at Tower 224 on 7 June 2008 is shown in Figure 7. The event began at 
0200 UTC and indicates a wave-like behavior in the time series that ended at 1300 UTC (denoted by the blue shaded 
box). During this time there were six oscillations in wind speed indicated by a change of 3 to 6 ms' 1 (6 to 12 kt) 
occurring approximately every 45 minutes. All of the model configurations correctly forecast the general trend of 
the wind speeds with the strongest occurring from 0000 to 0200 UTC then decreasing until 0800 UTC then 
increasing again until the end of the cycling event. Only the ADAS ARW configurations showed any indication of 
the wind speed cycling within the general trend. The ADAS ARW MYJ Eta configuration indicated four to five 
oscillations whereas the ADAS ARW Yonsei MM5 configuration indicated three oscillations within the event 
timeline. The MYJ Eta configuration also had larger magnitudes in the wind speed oscillation compared to the 
Yonsei MM5. This result was consistent for the other two towers for this event. 


A review of graphs at Tower 224 from a null case on 9 June 2007 is shown in Figure 8. Wind cycling was not 
observed at any of the EAFB towers on this day. The model was run from 0600 to 1800 UTC in the six 
configurations identical to the cycling events. During this time, the observed winds indicated a general weakening 
trend that all model configurations forecast. The LAPS NMM configurations indicated a low bias while the LAPS 
ARW and ADAS ARW indicated no overall bias and produced a better forecast than the LAPS NMM. 



Tower 224 7Jun08 ADAS ARW 

—Tower 224 -—MYJ Eta — Yonsei MMS 



Time (UTC) 


Figure 7. Graphs of a wind cycling case from Tower 224 on 7 June 2008. In each graph the observed steady-state 
wind speed is shown by the dark red line and the forecast steady-state wind speed is shown by the orange and blue 
lines representing various model configurations as identified by the graph legends. The light blue shaded box 
indicates the duration of the wind cycling event. 
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Figure 8. Graphs of a null case from Tower 224 on 9 June 2007. In each graph the observed steady-state wind 
speed is shown by the dark red line and the forecast steady-state wind speed is shown by the orange and blue lines 
representing various model configurations as identified by the graph legends. 


3.2 Model Indications of Wind Cycling Events 

None of the model configurations were able to accurately forecast the timing and magnitude of the steady-state 
wind speed oscillations at model grid points nearest to the wind towers. However, the model can still provide 
valuable information to the forecasters to indicate whether a wind cycling event may occur. The AMU compared 
plots of forecast steady-state winds over a large part of the model domain for all of the wind cycling events and the 
null cases to see if the model could discern between the event days and null days. 

Figure 9 shows four time intervals of model forecast steady-state winds plotted in the EAFB area for a wind 
cycling event day, 7 June 2008, from the ADAS ARW MYJ Eta configuration. The figure shows a progression of 
changing wind speed and direction near the concrete runway beginning with westerly winds of 7-10 kt at 0300 UTC 
(Figure 9a), then southwesterly at 10-15 kt at 0400 UTC (Figure 9b), changing to south-southeast at 1 0- 1 2 kt at 0500 
UTC (Figure 9c) and finally southeast at 1 5-20 kt at 0600 UTC (Figure 9d). As the northwesterly winds move across 
the Tehachapi Mountains and southern extent of the Sierra Nevada Mountains, they also move through the 
Tehachapi Pass that lies between the northeast end of the Tehachapi Mountains and the southern end of the Sierra 
Nevada Mountains (see Figure 1). During wind cycling events, the winds moving through the Tehachapi Pass will 
extend all the way to EAFB causing the wind speed and direction changes as the winds oscillate through the pass. 
The blue line in Figure 9 shows the approximate leading edge of the northwesterly winds moving across the 
mountains and through the pass. The winds moving through the pass cause the line to bulge eastward towards EAFB 
over time and disrupt the existing wind flow in the region. It is interesting to note that the Tehachapi Pass Wind 
Farm, located just east and south of the Tehachapi Pass, is one of California’s largest wind farms used for generating 
electricity (Edison International 2008). 
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Figure 9. Plots of ADAS ARW MYJ Eta model forecast steady-state winds for 7 June 2008 in the EAFB area for 
four model output times of 0300 UTC (a), 0400 UTC (b), 0500 UTC (c) and 0600 UTC (d). The location of EAFB 
is indicated by the white circle. The approximate location of the concrete runway is shown by the red line. The 
blue line indicates the leading edge of westerly winds. 


Figure 10 shows four time intervals of model forecast steady-state winds plotted in the EAFB area for a null 
case, 9 June 2007 from the ADAS ARW MYJ Eta configuration. The figure shows a nearly constant southwest wind 
flow with speeds of 5-10 kt throughout the series of images from 0700 to 1000 UTC near the concrete runway. The 
approximate leading edge of the northwesterly winds moving across the mountains and through the pass (blue line) 
was semi-stationary throughout the entire 12-hr model run. The northwesterly wind flow over the mountains was 
weaker in this null case compared to the cycling case shown in Figure 9, and the model correctly forecast a null 
event at EAFB. 
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Figure 10. Plots of ADAS ARW MYJ Eta model forecast steady-state winds for 9 June 2007 in the EAFB area 
for four model output times of 0700 UTC (a), 0800 UTC (b), 0900 UTC (c) and 1000 UTC (d). The location of 
EAFB is indicated by the white circle. The approximate location of the concrete runway is shown by the red line. 
The blue line indicates the leading edge of westerly winds moving across the Tehachapi Mountains. 

3.3 Subjective Analysis Summary 

Based on the subjective analysis of all wind cycling and null case days, the ARW runs outperformed the NMM 
runs. The NMM core consistently under-predicted wind speeds by the largest margin and did not capture changes in 
wind direction as well as the ARW core. Changing the model core seemed to have the biggest impact on the 
forecasts, while changing the model physics seemed to have the least impact on the forecasts. Although it appears 
the MYJ Eta scheme may have captured the cycling a bit better than the other configurations within the ARW core. 
The model did not forecast wind speeds better for the null days vs. the wind cycling days, but it clearly 
differentiated between cycling days and null cases used in this task. This would help the forecaster discern when a 
wind cycling case would occur. 
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4. Objective Analysis 

For the objective analysis, observed wind speed was compared to forecast wind speed using the latest version of 
the Model Evaluation Tools (MET) software. This software was developed by the NCAR Developmental Testbed 
Center. It is a state-of-the-art suite of verification tools that uses output from the WRF model to compute standard 
verification scores comparing gridded model data to point or gridded observations. Each statistic computed for this 
task compared the gridded WRF model data, available every 15 minutes, to the observations from the 12 wind 
towers at EAFB. In addition, the observed wind direction from the 12 wind towers was compared to forecast wind 
direction from the WRF model data. However, the MET software does not currently support objective analysis of 
wind direction. Therefore, the forecast wind direction was manually pulled from the model data and all statistics 
were computed using Microsoft Excel. 

Many output statistics are available within the MET software. This study looked at three of those statistics for 
wind speed: the forecast vs. the observed mean, the mean error, and the Pearson Correlation Coefficient (PCC); and 
two statistics for wind direction: the mean error and the PCC. The statistics compared all 12 towers combined to the 
corresponding locations in the model forecast output at a 15-minute interval. Towers that indicated wind cycling 
events were not separated from those that did not indicate wind cycling. Thus, the objective analysis is not a stand- 
alone analysis that indicates whether the wind cycling phenomena were captured, rather it only shows how well the 
model performed overall. The subjective analysis in Section 3 was used to determine if the model captured the wind 
cycling events. However, comparison of the objective analyses from the null cases to the wind cycling cases may 
help to determine whether the model is adept at forecasting a mesoscale process, such as wind cycling. 

The mean forecast vs. observed wind speed comparison shows how well the forecast wind speed corresponded 
to the observed wind speed. The mean forecast and observed wind speed, / and o , respectively, were computed as 
follows: 


f=-m) 



where: 

n = 12 for the number of tower locations, 
f = forecast wind speed at a tower location, and 
o, = observed wind speed at a tower. 

The mean error, ME, is a measure of the overall bias of the wind speed or direction. A perfect forecast has ME = 
0. It is defined as: 


n m 

where n,f, and o, are defined as above and: 

/ = average forecast wind speed/direction at 1 2 tower locations, and 

o = average observed wind speed/direction at 12 towers. 

The PCC, r, measures the strength of the linear association between the forecast and observed wind speed or 
direction. It is defined as: 
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±(f,-7h-°) 


1=1 



where 

f = forecast wind speed/direction at all tower locations for one output period, 

o, = observed wind speed/direction at all tower locations for one output period, 

/ = average forecast wind speed/direction for 12-hour forecast, and 

o = average observed wind speed/direction for 12-hour forecast. 

The PCC can range between -1 and 1; 1 indicates a perfect correlation, -1 indicates a perfect negative correlation, 
and 0 indicates no correlation between the forecast and observations. Specifically, the PCC for wind speed measures 
whether large values of forecast wind speed tend to be associated with large values of observed wind speed (positive 
cofrelation); whether snrail vaiues of fbrecast wind speed tend to be associated with targe values of observed wind 
speed or vice versa (negative correlation), or whether values of both variables are unrelated (correlation near 0). For 
wind direction, the PCC measures the rotation of the wind with a perfect positive correlation indicating that the 
winds are shifting in the same direction with the same magnitude and a perfect negative correlation indicating that 
the winds are shifting in opposite directions with the same magnitude. 

4.1 Forecast vs. Observed Wind Speed 

All WRF model configurations under-predicted the wind speed throughout the forecasts for all wind cycling 
case days, except for the 30/3 1 July 2008 run, which was the marginal wind cycling case day. Figure 1 1 shows the 
forecast vs. observed mean wind speed for three of those wind cycling case days: 22 December 2006 (Figure 1 la), 
4/5 March 2008 (Figure 1 lb), and 30/31 July 2008 (Figure 1 1c), and one null case: 9 June 2007 (Figure 1 Id). As 
can be seen from Figure 1 la and b, the only model configuration that did not under-predict the wind speed was the 
LAPSARWMYJ on 4/5 March (Figure 1 lb). For 30 July (Figure 1 lc), all model configurations over-predicted the 
wind speed, except for the LAPSARW_Yonsei configuration. For the 9 June null case (Figure lid), the model 
slightly under-predicted the wind speed. 
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Figure 1 1 . Chart of the forecast mean wind speed from the six WRF model configurations vs. the observed mean 
wind speed from the EAFB towers for the 12-hour forecast period on a) 22 December 2006, b) 4/5 March 2008, c) 
30 July 2008, and d) 9 June 2007. 


4.2 Mean Error for Wind Speed and Direction 

As stated above, the mean error is a measure of the overall bias of the wind speed or direction. Figure 12 shows 
the mean error for wind speed for the same three wind cycling case days and one null case as in Figure 1 1. Again, it 
is apparent that all model configurations generally under-predicted the wind speed for both the 22 December (Figure 
12a) and 4/5 March (Figure 12b) cases. In both cases, the LAPSNMMGFS was the worst performer, under- 
predicting the wind speed by 4 to 8 m/s throughout the forecast. Figure 12a reveals that the only model 
configurations to over-predict wind speed during the forecast were both of the ADAS runs. By the second half of the 
forecast, all model configurations except the LAPSNMMGFS had nearly the same mean error values. Figure 12b 
shows that all of the ARW runs were clustered throughout the entire 12-hour forecast, indicating that they captured 
and missed all of the same fluctuations in the observed wind speed. The mean error for 30/31 July (Figure 12c) 
shows the same range in mean error values as for 22 December and 4/5 March, but reversed. That is, in general the 
model over-predicted the wind speeds throughout the forecast with the mean error falling in the positive range. 
Unlike the other two wind cycling cases, the NMM runs outperformed all ARW runs with the LAPSNMM MYJ 
performing the best and the ARW MYJ runs performing the worst. For the null case (Figure 12d), the latter half of 
the forecast was more accurate than the beginning of the forecast. The mean error range for this case was smaller 
than for the wind cycling cases, but this was expected since the wind speed values were half of those for the wind 
cycling days. A comparison of the bias, given below, will give a more accurate comparison of the wind cycling days 
to the null cases. 
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Figure 12. Chart of the mean error for wind speed from the six WRF model configurations for the 12-hour 
forecast period for a) 22 December 2006, b) 4/5 March 2008, c) 30 July 2008, and d) 9 June 2007. 

Table 4 lists the average mean error and bias for wind speed for the entire 12-hour forecast for each model 
configuration for each wind cycling case day and the two null cases. The smallest average mean error/bias is 
highlighted in red for each case day and the largest average mean error/bias is highlighted in blue. Overall, both the 
LAPSARWMYJ and ADASARW MYJ produced the smallest average mean errors and had the smallest bias of all 
configurations, followed by the ADASARW Yonsei configuration. Clearly the worst performer among all model 
configurations was the LAPSNMMGFS which was the worst in seven out of the nine cases. 


Table 4. Average mean error and bias (parenthesis) for the 12-hour forecast for each model configuration for the wind 
cycling and null cases. The smallest average mean error/bias is highlighted in red for each case day. 


Model Configurations 

Date 

LAPSARW 

Yonsei 

LAPSARW 

MYJ 

ADASARW 

Yonsei 

ADASARW 

MYJ 

LAPSNMM 

GFS 

LAPSNMM 

MYJ 

22 Dec 06 

-1.96 (0.77) 

-2.28 (0.93) 

-1.72(1.03) 

-1.43 (1.09) 

-4.58 (0.53) 

-3.14(0.75) 

30 Jan 08 

-2.86 (0.79) 

-1.81 (0.88) 

-3.43 (0.74) 

-2.25 (0.84) 

-7.45 (0.39) 

-5.78 (0.54) 

14 Feb 08 

-1.73 (0.86) 

-0.79 (0.97) 

-2.62(0.71) 

-2.7(0.68) 

-5,1 (0.48) 

-3.25 (0.69) 

4/5 Mar 08 

-1.23 (0.82) 

-0.6 (0.89) 

-1.71 (0.75) 

-1.19(0.81) 

- 4.96 ( 0 . 35 ) 

-3.77(0.49) 

4 Jun 08 

-3.52 (0.77) 

-2.57(0.84) 

-2.5 (0.83) 

- 1.1 ( 0 , 95 ) 

-/. 44 (0,4 j) 

-5.69 (0.58) 

7 Jun 08 

-1.17(0.88) 

-0.75 (0.92) 

-0.94 (0.93) 

- 0.75 ( 0 . 94 ) 

” 4.1 ( 0 , 54 ) 

-2.33 (0.75) 

30/31 Jul 08 

1.64(1.23) 

3 . 58 ( 1 . 5 ) 

2.01 (1.28) 

3.21 (1.45) 

-1.68 (0.76) 

- 0.2 ( 0 . 97 ) 

9 Jun 07 (null case) 

-0.58 (0.92) 

-0.99 (0.78) 

0.3 ( 0 . 99 ) 

-0.31 (0.82) 

- 1.91 ( 0 . 47 ) 

-1.67 (0.53) 

1 Nov 07 (null case) 

-0.18(0.86) 

0.17(1.12) 

- 0.5 ( 0 . 66 ) 

0 . 05 ( 1 . 05 ) 

-0.36 (0.77) 

-0.36 (0.77) 
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Figure 13 shows the mean error for wind direction for the same three wind cycling case days and one null case. 
In general, the model configurations that used the same cores but different physics give nearly the same mean error 
for all wind cycling and null cases. In fact, all model configurations seemed clustered throughout the forecast, 
indicating that they captured and missed all of the same shifts in wind direction. Overall, no one model 
configuration was the best or worst performer for a majority of the cases. The discrepancies between model 
configurations for mean error of wind direction were smaller than for the wind speed, indicating that the model may 
have been better at forecasting wind direction than wind speed. In the highlighted cases, the smallest mean error 
occurred on 30/31 July (Figure 13c), while the largest mean error occurred for the null case on 9 June (Figure 13d). 
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Figure 13. Chart of the mean error for wind direction from the 6 WRF model configurations for the 12-hour 
forecast period for a) 22 December 2006, b) 4/5 March 2008, c) 30 July 2008, and d) 9 June 2007. 


4.3 Pearson Correlation Coefficient for Wind Speed and Direction 

The PCC can indicate whether the model configurations caught the overall trend of the wind speed. That is, it 
answers the question of whether the model winds fluctuated positively and negatively with the same magnitude as 
the observed winds. The closer the correlation coefficient is to 1, the better the model was able to capture these 
trends. When comparing forecast vs. observed wind speed, only positive coefficients indicate any value in the model 
forecasts. 

Figure 14 shows the PCC for wind speed from the 12-hour forecasts of all wind cycling and null cases 
simulated with the six model configurations. The most obvious feature is that on both 30 January 2008 and 4 June 
2008 each model configuration did a poor job at capturing the trends in the observed winds. The best model forecast 
days were 4/5 March 2008 and 30/3 1 July 2008. Both days had correlation coefficients of 0.6 or above for all model 
configurations. The best perfonner for 4/5 March was the LAPSNMMMYJ and the LAPSNMM_GFS for 30/31 
July; however, the bias for these two configurations was poor compared to the other configurations for the same day. 
This indicates that although the model forecast wind speeds w'ere too low, the model configuration was able to 
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capture the fluctuations in wind speed maximums and minimums. Disregarding the PCCs for the two worst days (30 
January 2008 and 4 June 2008), the most consistently good performers were the LAPSARWYonsei and 
LAPSNMM MYJ configurations. 
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Figure 14. Chart showing the Pearson Correlation Coefficient for wind speed for the 12-hour forecasts for 
all wind cycling and null cases for the six model configurations. 

For wind direction, the PCC can indicate whether the model configurations caught the overall shifts in wind 
direction. That is, it answers the question of whether the model wind direction fluctuated positively and negatively 
with the same magnitude as the observed wind direction. The closer the con-elation coefficient is to 1, the better able 
the model to capture the same magnitude and direction of the observed wind shift. A correlation coefficient closer to 
-1 indicates that the model was able to capture the magnitude of the wind shift, but in the opposite direction. 

Figure 15 shows the PCC of wind direction for the 12-hour forecasts for all wind cycling and null cases for the 
six model configurations. Unlike the PCC for the wind speed forecasts, each model configuration did a poor job at 
capturing any of the trends in the observed winds on 30/31 July 2008. In fact, the average PCC for this day for all 
model configurations combined is 0, or no correlation. The model configurations did well capturing the shifts in 
wind direction for the rest of the wind cycling case days. Overall, the model was better able to capture shifts in wind 
direction for the wind cycling days than for the null cases. The best model forecast day was 7 June 2008, which had 
a correlation coefficient of 0.6 or above for all model configurations. As was found with the mean error calculations, 
there is no indication of one model configuration as clearly the best or worst performer. 
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Figure 15. Chart showing the Pearson Correlation Coefficient for wind direction for the 12-hour forecasts 
for all wind cycling and null cases for the six model configurations. 

4.4 Objective Analysis Summary 

Based on the objective analysis of all wind cycling and null case days, it appears that overall the ARW runs 
outperformed the NMM runs. Although the NMM core was more successful in capturing the increases and decreases 
in wind speed, it did a poor job at capturing the magnitude of the wind and consistently under-predicted wind speeds 
by the largest margin. Changing the model core seemed to have the biggest impact on the forecasts, while changing 
the model physics seemed to have the least impact on the forecasts. It does not appear the model does any better 
forecasting wind speeds for the null days vs. the wind cycling days. 


27 


5. Conclusions 

Occasionally, the space shuttle must land at EAFB in Southern California when weather conditions at KSC 
violate Flight Rules. This can create a challenge for SMG as the complex terrain in and around EAFB makes 
forecasting surface winds difficult. In particular, “wind cycling” cases, in which the wind speeds and directions 
oscillate among towers near the EAFB runway, present a challenging forecast problem for shuttle landings. An 
accurate depiction of the winds along the runway is crucial in making the landing decision. Since global and national 
scale models cannot properly resolve the wind field due to their coarse horizontal resolutions a properly tuned high- 
resolution mesoscale model, like WRF, is needed. 

The evaluation described in this report was designed to assess the skill of different WRF model configurations 
for forecasting wind cycling events at EAFB. This study was done in two steps: 1) Initializing both cores of the 
WRF model with either the LAPS or ADAS software, while also varying some of the physical parameters, and 2) 
Assessing the accuracy of model output by comparing surface wind forecasts to local EAFB tower observations. 

SMG identified seven wind cycling case days from December 2006 to July 2008. Six of the seven days were 
considered strong wind cycling cases, while one was a marginal case. The AMU also randomly chose two null 
cases. To compare the different model configurations, the WRF model was run for 12 hours at a 1-km horizontal 
grid spacing over EAFB and the surrounding mountainous areas using data from the identified case days. Data 
ingested by-the model through The data analysis packages -included- Level MI WSR'SSD data, GOES visible and 
infrared satellite imagery, MADIS data, and local EAFB wind tower data. Each candidate day consisted of six runs 
using different model configurations, for a total of 42 runs for the wind cycling case days and 14 for the null cases. 
The different configurations included: 

• LAPS-ARW with the Yonsei University PBL scheme and MM5 similarity surface layer scheme, 

• LAPS-ARW with the Mellor-Yamada-Janjic PBL scheme and ETA similarity surface layer scheme, 

• LAPS-NMM with the NCEP GFS PBL scheme and NCEP GFS surface layer scheme, 

• LAPS-NMM with the Mellor-Yamada-Janjic PBL scheme and ETA similarity surface layer scheme, 

• ADAS-ARW with the Yonsei University PBL scheme and MM5 similarity surface layer scheme, and 

• ADAS-ARW with the Mellor-Yamada-Janjic PBL scheme and ETA similarity surface layer scheme. 

5.1 Results 

The major results from the evaluation show the following: 

• Subjective analysis, 

o Overall, the ARW runs outperformed the NMM runs. 

o Changing the model core seemed to have the biggest impact on the forecasts, while changing the 
model physics seemed to have the least impact on the forecasts. 

o Model configurations that used the MYJ PBL scheme seemed to slightly outperform other PBL 
schemes. 

o The model was able to differentiate between wind cycling days and null cases. 

• Objective analysis, 

o Overall the ARW runs outperformed the NMM runs. 

o The NMM core was more successful in matching the increasing and decreasing trends in wind speed 
with the observations, but did a poor job at capturing the magnitude of the wind and consistently 
under-predicted wind speeds by the largest margin. 

o As was found in the subjective analysis, changing the model core seemed to have the biggest impact 
on the forecasts, while changing the model physics seemed to have the least impact on the forecasts. 

o It does not appear that the model did any better forecasting wind speeds for the null days vs. the wind 
cycling days. 

o Model configurations that used the MYJ PBL scheme seemed to slightly outperform other PBL 
schemes. 
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In both the subjective and objective analyses, the AMU found the ARW core and MYJ PBL scheme performed 
better than the other model configurations. The model did not produce a better wind forecast on null days vs. wind 
cycling days. The model was able to differentiate between wind cycling days and null cases which would provide 
added value to the shuttle landing forecast. 

5.2 Recommendations 

Of the six model configurations tested, the AMU recommends either the ADAS ARW MYJ or LAPS ARW 
MYJ configuration for operational use for predicting wind cycling events at EAFB for the following reasons: 

• Both configurations consistently had the lowest bias for wind speed compared to the other model 
configurations, 

• Both configurations best captured wind speed oscillations when compared to the observed wind speeds at 
the concrete runway towers, and 

• The NMM core produced wind speed forecasts well below those observed and did not capture the wind 
direction changes well on the cycling days. 

.... 5.3 Future, Wgrk , 

The goal of this work was to assess different WRF model options and to determine how well the model could 
predict surface wind speed and direction at EAFB and if one model configuration performed better than others. 
While that goal was met, the data indicated further investigations outside the scope of this task could include 
sophisticated signal analysis, rotating the model coordinate systems orthogonal and/or parallel to the prevailing wind 
flow and closely comparing the observed and forecast wind speed cycles to wind direction cycles which may lead to 
discoveries of why the winds cycle as they do and how that relates to model configuration and performance. 
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